In recent years, we have greeted the age of multimedia in which audio, video, and other data are integrally handled. In parallel with conventional information media, such as newspaper, magazine, television, radio, and telephone, multimedia has been adopted as means for transmitting information to man. Generally, "multimedia" means to show, not only characters, but also diagrams, voices, and especially images, simultaneously, in relation with each other. In order to handle the above-described information media as objects of multimedia, it is necessary to change the information into a digital format.
Generally, digitized information hardly deteriorates although it is repeatedly recorded or transmitted. This is of great advantage to the user, but is not always desirable for the provider of the information. That is, when there is a malicious third party who illegally duplicates the provided information (hereinafter, referred to simply as "third party"), the third party can easily get the same information as the provided one and, therefore, it is difficult for the provider to claim a right to, for example, have a recompense for his/her providing the information.
In order to avoid such illegal duplication, a variety of methods for recording special information (hereinafter, referred to as "secret information") in information to be provided, have been proposed. One of the methods is to add secret information to a header or the like, and this method is used for music CDs or the like. According to this method, however, the secret information is easily altered by the third party.
On the other hand, when secret information is multiplexed into information itself, the third party cannot easily distinguish between the secret information and the original information. In this case, there is a great possibility that the secret information is duplicated while it is protected. When the user, for example, the user of an information decoder, tries to reproduce information, if the information decoder extracts the secret information and detects from the secret information that the information to be reproduced is an illegally duplicated one, the decoder informs the user that this information is an illegal duplication, whereby the user can stop display or output of the information. Adopted as secret information is as follows: information for specifying the author of original information, or information showing the right to duplicate, i.e., whether duplication is permitted or not.
Now, a description is given of the case where the third party illegally gets information in the final stage. Assuming that an image signal is supplied as information, the third party may illegally get the information from the input signal to a display unit. To be specific, since the third party can take the information displayed on the display unit as an electric signal, in order to prevent the secret information from being deleted even in such case, it is desirable that the secret information is multiplexed in the information displayed on the display unit. In this case, it is an indispensable condition that the secret information should be multiplexed so that it does not visually interfere with the displayed information on the display.
FIGS. 21(a)-21(d) are diagrams for explaining an example of a method for multiplexing secret information. FIG. 21(a) shows an image signal to be transmitted or recorded, and FIG. 21(b) shows a specific pattern showing secret information. Initially, this pattern is converted using specific mapping function or spread spectrum to distribute data over a screen. Thereby, the secret information is converted to a signal that does not cause visual interference, as shown in FIG. 21(c). Generally, this conversion is carried out by the spread spectrum method. From the signal shown in FIG. 21(c), it is difficult to analyze what sort of secret information is inserted, like a cipher.
The secret information converted as shown in FIG. 21(c) is added to the image signal shown in FIG. 21(a), whereby FIG. 21(d) is obtained as image information to be transmitted or recorded actually. When the recorded or transmitted image information is regenerated or received, the secret information can be restored by extracting specific data from the image information shown in FIG. 21(d) according to the inverse of the rule on the insertion side.
For example, the image information shown in FIG. 21(d) is subjected to special arithmetic processing, for example, averaging or calculation of correlation coefficients with a specific pattern, to extract the feature of FIG. 21(b), that is, a transverse band in the center of the screen, whereby secret information corresponding to this feature (transverse band) is obtained. Even though the method of extracting the secret information from FIG. 21(d) has already been known, if FIG. 21(a) or FIG. 21(c) is indecipherable from FIG. 21(d), the third party cannot remove the secret information from the image information shown in FIG. 21(d).
Furthermore, when the third party makes such interference with the image information shown in FIG. 21(d) that the secret information cannot be extracted, if the image degradation exceeds a tolerance, the third party cannot obtain an image as shown in FIG. 21(a). In this case, the image information is protected from the intentional interference by the third party.
FIG. 22(a) is a block diagram illustrating a conventional information embedding apparatus. This information embedding apparatus comprises an information converter 12 and a compositor 14. Secret information 10 is input to the information converter 12 while an image signal 11 is input to the compositor 14. The information converter 12 converts the secret information 10 to a pattern corresponding to the secret information as shown in FIG. 21(b) and, further, it converts the pattern to a signal that does not cause visual interference. The compositor 14 multiplexes the converted secret signal 13 in the image signal 11, thereby generating an image signal 15 for recording or transmission.
FIG. 22(b) is a block diagram illustrating a conventional information extracting apparatus. This information extracting apparatus comprises an extractor 20 and an information converter 22. The extractor 20 extracts pattern information 21 relating to the secret signal from the input image signal 15. Receiving the pattern information 21, the information converter 22 converts it to secret information corresponding to the pattern and outputs the secret information.
In the conventional information embedding method described above, however, the secret information is easily destroyed sometimes. FIGS. 23(a)-23(d) are diagrams for explaining the case where the secret information is destroyed. FIG. 23(a) shows an image obtained by subjecting the recorded or transmitted image shown in FIG. 21(d) to parallel translation and clipping, wherein the absolute position of the image including a piano and a trumpet is shifted toward the upper left.
In the conventional information embedding method, since embedding and extraction of secret information are carried out on the basis of the absolute position of image, when the image is parallel-translated as described above, the secret information is detected as a pattern as shown in FIG. 23(b) which is completely different from the pattern shown in FIG. 21(b). For example, when an image is duplicated by an analog device of which precision is not very high, because of disagreement of synchronous signals, the absolute position of the image sometimes shifts toward right or left. In this case, there is a possibility of damaging the secret information even when the user unconsciously operates the device.
Furthermore, the third party can destroy the secret information by making the embedded information shown in FIG. 21(c) undetectable. This is realized by adding an interference signal as shown in FIG. 23(c) to the image. When the absolute position of the embedded information is fixed by the producer of the image, the third party can easily generate such an interference signal.
Furthermore, in the conventional information embedding method, since only one kind of secret information is embedded, it is only detected whether the secret information is recorded or not. From the result of this detection, it cannot be known whether illegal interference is added or not.